Monitoring and tracking system, method, article and device

ABSTRACT

A monitoring system includes one or more monitoring devices configured to detect conditions related to a monitored person and an artificial intelligence module configured to analyze historical information related to conditions of the monitored person and current information related to conditions of the monitored person and determine whether to generate an alert signal based on the analysis. The artificial intelligence module may also be configured to update the historical information related to the conditions of the monitored person based on the current information and responses to the alert signals. The system includes a moveable sensor and a monitored-person identification subsystem.

BACKGROUND Technical Field

The present disclosure relates to systems, methods and articles formonitoring and tracking systems.

Description of the Related-Art

Monitoring and tracking systems may employ a manual or mechanicallytriggered signal to indicate a monitored premises or a monitored personis in danger or needs assistance. For example, a button on a necklacemay be pressed to generate a signal indicating a person needsassistance, or a window may be fitted with a detector to detect openingof the window. The signal may trigger a remote alarm.

BRIEF SUMMARY

In an embodiment, a monitoring system comprises: at least one deviceconfigured to generate signals indicative of a safety status of anindividual; and a safety status monitoring module configured to receivethe signals indicative of the safety status of the individual and todetermine based on the received signals whether to generate a safetyalert regarding the safety status of the individual and, when it isdetermined to generate a safety alert, to cause the safety alert to begenerated and transmitted. In an embodiment, the at least one deviceincludes one or more devices selected from: a location sensor; a thermalsensor; a health status monitoring device; an image capture device; anda video capturing device. In an embodiment, the at least one device iscommunicatively coupled with remote servers configured to implement thesafety status monitoring module. In an embodiment, the at least onedevice includes a device configured to perform voice communications. Inan embodiment, the device configured to perform voice communications isconfigured to be activated in response to a determination to activate asafety alert.

In an embodiment, a method comprises: monitoring indications of a safetycondition of an individual; and generating a safety alert based on themonitored indications. In an embodiment, the monitoring indicationsincludes monitoring at least one device selected from: a locationsensor; a thermal sensor; a health status monitoring device; and a videocapturing device. In an embodiment, the at least one device iscommunicatively coupled with remote servers configured to perform themonitoring and the generating. In an embodiment, the method comprisesinitiating voice communications in response to a safety alert. In anembodiment, a non-transitory computer-readable medium contents areconfigured to cause one or more devices to perform one or more methodsdisclosed herein.

In an embodiment, a system comprises: means for monitoring indicationsof a safety condition of an individual; and means for generating asafety alert based on the monitored indications.

In an embodiment, a safety monitoring system comprises: one or moresensors configured to generate at least one signal indicative of acurrent condition related to a safety status of a person; and one ormore processing devices configured to: determine a safety status of theperson based on the at least one signal indicative of the currentcondition and stored information related to the safety status of theperson; update the stored information related to the safety status ofthe person based on the at least one signal indicative of the currentcondition; and when the determined safety status indicates the personmay be in danger: initiate one or more actions based on the determinedsafety status; monitor responses to the one or more initiated actions;and update the stored information related to the safety status of theperson based on the monitored responses. In an embodiment, the one ormore initiated actions comprise one or more of: generating an alertsignal to the person; transmitting a signal to a remote server; andgenerating an alert message based on stored contact information. In anembodiment, the one or more sensors include at least one of: a locationsensor; a thermal sensor; a health status monitoring device; a motionsensor; a facial recognition sensor; and a video capturing device. In anembodiment, a sensor may sense one or more of a temperature, a pulse, abreathing state (e.g. breathing rate, breathing depth), a heart rate,motion, objects, images, etc. In an embodiment, the safety monitoringsystem comprises a device configured to perform voice communications. Inan embodiment, the stored information comprises property identifiersindicative of conditions and the one or more processing devices comprisean artificial intelligence module configured to compare the at least onesignal to the stored property identifiers and to determine the safetystatus of the person based on the comparison. In an embodiment, thedetermining the safety status comprises determining a position of aproperty identifier on a characteristic property-identifier curve basedon the at least one signal. In an embodiment, the curve is a Bell curve.In an embodiment, the characteristic Bell curve is based on storedproperty identifiers indicative of the characteristic. In an embodiment,the characteristic Bell curve is related to one or more of: an objectidentifier; a location identifier; a position identifier; a timeidentifier; a sound identifier; a motion identifier; a physical statusidentifier; and an emotional identifier. In an embodiment, thedetermining the safety status comprises determining whether the positionis within one or more threshold deviations from a mean of the Bellcurve. In an embodiment, the stored information includes an initial dataset and updating the stored information comprises adding data to theinitial data set. In an embodiment, the initial data set is a genericdata set not specific to the person. In an embodiment, one or more ofthe sensors may be mounted on a mobile device (e.g., a robot), which, inoperation, identifies a person to monitor (e.g., using one or moresensors and stored identification information, such as facialrecognition data, physical shape data, other personal data) and followsthe identified monitored person as the monitored person moves around. Inan embodiment, the robot may have a defined area in which to follow themonitored person around (e.g., inside a house or an apartment).

In an embodiment, the monitoring system includes a camera which, inoperation, follows a monitored body around a house. In an embodiment,the camera uses a wide angle lens. In an embodiment, the system settingsare selected (such as a lens to use, focus settings, sensing timeframes) to use based on monitoring conditions or desired characteristicsto monitor (e.g., distance to the monitored person, data to be sensed(e.g., temperature, body position, breathing rate, breathing depth,heart rate, etc.)). In an embodiment, the camera is mounted to amoveable support (e.g., a robot), which, in operation, moves or followsthe monitored person based on detected motion and/or other information.

In an embodiment, the system comprises one or more motion sensors andone or more thermal sensors, wherein in operation, the thermal sensorsare activated and deactivated based on information sensed by the motionsensors. In an embodiment, the system comprises one or more motionsensors and one or more image sensors, wherein in operation, the imagesensors are activated and deactivated based on information sensed by themotion sensors.

In an embodiment, the system identifies a person to be monitored basedon one or more of facial recognition techniques, body temperature, bodytype/shape, other biological indicators, etc. This facilitatesidentifying a person to monitor when more than one person is in amonitored location. For example, a house having more than one occupantwhen only one occupant is to be monitored (e.g., an occupant havingdiabetes or other health issues who may be unable to summon assistancein a medical emergency).

In an embodiment, the system includes a robot which follows a person tobe monitored (e.g., a senior, a patient) and includes one or more of animage sensor which senses images at a wide angle or selectable angles,an infrared sensor, infrared beams, infrared night vision systems,facial recognition subsystems, etc. to facilitate identifying andmonitoring the person to be monitored. In an embodiment, the system mayprovide medication to the patient, monitor patient medication intake,measure blood pressure, measure glucose content, measure heart rate,etc. (with or without using a robot in an embodiment having a robot).

In an embodiment, a system comprises a combination of mounted sensors(e.g., a camera or motion sensor mounted on a wall) and mobile sensors(e.g., one or more sensors carried by a person to be monitored, one ormore sensors mounted to one or more robots). For example, an entry wayor staircase may have a mounted sensor, a first floor may have a firstrobot, a second floor may a second robot, etc., and various combinationsthereof.

In an embodiment, a method comprises: receiving at least one signalindicative of a current condition related to a safety status of aperson; determining, using at least one processing device, a safetystatus of the person based on the at least one signal indicative of thecurrent condition and stored information related to the safety status ofthe person; updating, using the at least one processing device, thestored information related to the safety status of the person based onthe at least one signal indicative of the current condition; and whenthe determined safety status indicates the person may be in danger:initiating one or more actions based on the determined safety status;monitoring responses to the one or more initiated actions; and updatingthe stored information related to the safety status of the person basedon the monitored responses. In an embodiment, the one or more initiatedactions comprise one or more of: generating an alert signal to theperson; transmitting a signal to a remote server; and generating analert message based on stored contact information. In an embodiment, theat least one signal includes at least one of: a signal indicative of alocation of the person; a signal indicative of a temperature; a signalindicative of a health status of the person; and an imaging signal. Inan embodiment, the stored information comprises property identifiersindicative of conditions and the at least one processing devicecomprises an artificial intelligence module configured to compare the atleast one signal to the stored property identifiers and to determine thesafety status of the person based on the comparison. In an embodiment,the determining the safety status comprises determining a position of aproperty identifier on a characteristic Bell curve based on the at leastone signal. In an embodiment, the characteristic Bell curve is based onstored property identifiers indicative of the characteristic. In anembodiment, the characteristic Bell curve is related to one or more of:an object identifier; a location identifier; a position identifier; atime identifier; a sound identifier; a motion identifier; a physicalstatus identifier; and an emotional identifier. In an embodiment, anon-transitory computer-readable medium contents configure a safetymonitoring system to perform a method, the method comprising: receivingat least one signal indicative of a current condition related to asafety status of a person; determining a safety status of the personbased on the at least one signal indicative of the current condition andstored information related to the safety status of the person; updatingthe stored information related to the safety status of the person basedon the at least one signal indicative of the current condition; and whenthe determined safety status indicates the person may be in danger:initiating one or more actions based on the determined safety status;monitoring responses to the one or more initiated actions; and updatingthe stored information related to the safety status of the person basedon the monitored responses.

In an embodiment, a system comprises: means for generating at least onesignal indicative of a current condition related to a safety status of aperson; means for determining a safety status of the person based on theat least one signal indicative of the current condition and storedinformation related to the safety status of the person; means forupdating the stored information related to the safety status of theperson based on the at least one signal indicative of the currentcondition; and means for, when the determined safety status indicatesthe person may be in danger, initiating one or more actions based on thedetermined safety status; monitoring responses to the one or moreinitiated actions; and updating the stored information related to thesafety status of the person based on the monitored responses.

In an embodiment, a safety monitoring system comprises: a plurality ofsensors configured to generate signals indicative of a current conditionrelated to a safety status of a person; and one or more processingdevices coupled to the plurality of sensors and configured to: determinea safety status of the person based on the signals indicative of thecurrent condition and stored information related to the safety status ofthe person; update the stored information related to the safety statusof the person based on the signals indicative of the current condition;and in response to the determined safety status indicating the personmay be in danger: initiate one or more actions based on the determinedsafety status; monitor responses to the one or more initiated actions;and update the stored information related to the safety status of theperson based on the monitored responses. In an embodiment, the one ormore initiated actions comprise: generating an alert signal to theperson; transmitting a signal to a remote server; generating an alertmessage based on stored contact information; or combinations thereof. Inan embodiment, the plurality of sensors include: a location sensor; athermal sensor; a health status monitoring device; a radar sensor; avideo capturing device; or combinations thereof.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

FIG. 1 is a functional block diagram of an embodiment of an environmentsuitable for providing monitoring and tracking services.

FIG. 2 is a functional block diagram of an embodiment of an environmentsuitable for providing monitoring and tracking services.

FIG. 3 is a flow diagram of a method of monitoring a location of aperson.

FIG. 4 is a functional block diagram of an embodiment of an environmentsuitable for providing monitoring and tracking services.

FIG. 5 is a flow diagram of a method of monitoring a health condition ofa person.

FIG. 6 is a functional block diagram of an embodiment of an environmentsuitable for providing monitoring and tracking services.

FIG. 7 is a flow diagram of a method of monitoring a person based onthermal images.

FIG. 8 is a functional block diagram of an embodiment of an environmentsuitable for providing monitoring and tracking services.

FIG. 9 is a flow diagram of a method of monitoring a person based onvideo images.

FIG. 10 is a functional block diagram of an embodiment of an environmentsuitable for providing monitoring and tracking services.

FIG. 11 is an illustration of an example property-identifier curve.

FIG. 12 illustrates example zone and spot based temperature curves.

FIG. 13 illustrates an embodiment of a system and an embodiment of amethod of operating a system, such as the various systems describedherein, to monitor an individual.

FIG. 14 is an illustration of monitoring breathing rate using remotesensing technology.

FIG. 15 is a functional block diagram of an embodiment of an environmentsuitable for providing monitoring and tracking services.

DETAILED DESCRIPTION

In the following description, certain details are set forth in order toprovide a thorough understanding of various embodiments of devices,systems, methods and articles. However, one of skill in the art willunderstand that other embodiments may be practiced without thesedetails. In other instances, well-known structures and methodsassociated with, for example, mobile devices such as smart phones, GPSdevices and systems, computing systems, telecommunication networks, webbrowsers, web servers, etc., have not been shown or described in detailin some figures to avoid unnecessarily obscuring descriptions of theembodiments.

Unless the context requires otherwise, throughout the specification andclaims which follow, the word “comprise” and variations thereof, such as“comprising,” and “comprises,” are to be construed in an open, inclusivesense, that is, as “including, but not limited to.”

Reference throughout this specification to “one embodiment,” or “anembodiment” means that a particular feature, structure or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. Thus, the appearances of the phrases “in one embodiment,” or“in an embodiment” in various places throughout this specification arenot necessarily referring to the same embodiment, or to all embodiments.Furthermore, the particular features, structures, or characteristics maybe combined in any suitable manner in one or more embodiments to obtainfurther embodiments.

The headings are provided for convenience only, and do not interpret thescope or meaning of this disclosure or the claimed invention.

The sizes and relative positions of elements in the drawings are notnecessarily drawn to scale. For example, the shapes of various elementsand angles are not drawn to scale, and some of these elements areenlarged and positioned to improve drawing legibility. Further, theparticular shapes of the elements as drawn are not necessarily intendedto convey any information regarding the actual shape of particularelements, and have been selected solely for ease of recognition in thedrawings.

Embodiments of systems and methods in which electronic devices such asmonitoring and tracking devices are described herein.

The following discussion provides a brief, general description of asuitable computing environment in which the embodiments described hereinmay be implemented. Although not required, various embodiments will bedescribed in the general context of computer-executable instructions,such as program application modules, objects, or macros being executedby one or more electronic devices, such as a monitoring and trackingdevice, a smart phone, a personal computer, a server, etc., and variouscombinations thereof. Those skilled in the relevant art will appreciatethat various embodiments can be practiced with other computing systemconfigurations, including other handheld devices, multiprocessorsystems, microprocessor-based or programmable consumer electronics,networked personal computers (PCs), minicomputers, mainframe computers,virtual systems, and the like. Various embodiments can be practiced indistributed computing environments where tasks or modules are performedby remote processing devices, which are linked through a communicationsnetwork. In a distributed computing environment, program modules may belocated in both local and remote memory storage devices.

For people with mobility difficulties due to disability, injury, illnessor cognitive impairment (Alzheimer's, dementia, memory loss, etc.),daily functioning can be a constant challenge, with serious risks andpotentially deadly consequences. Naturally, friends and relatives ofthese people are concerned for their welfare and would like to know whenhelp is needed.

An embodiment of a monitoring system provides a tracking and monitoringrecord and transmits information to the concerned relatives/parties. Inan embodiment, an alert is generated and provided to designatedentities, such as friends and families (concerned parties) or devicesassociated with such parties when there is an indication the monitoredperson is in a situation where assistance might be needed. These alertscan be sent to any devices that are capable of receiving electronicmessages, such as computers (all kinds), cell phones, hand held devices,personal call/voice messages, pagers, etc. Concerned parties can alsouse any digital device to access the secure record from a datacenter, oruse the Internet to access the status record of the monitored person.

In an embodiment, the monitoring system comprises various components,such as:

A) Geo system—a sensing device that tracks the monitored person'slocation or position;

B) Health tracking system—monitors changes in one or more of theperson's basic vital signs;

C) Thermal system—a thermal image monitoring system (infrared sensor,infrared beams, infrared night vision subsystems, etc.);

i) an AI image recognition system to analyze and determine the status ofthe human “object” (e.g., sitting, standing, reading, fighting, fallen,crawling on the floor, etc.);

ii) a human monitoring group to determine the status of the monitoredperson;

D) Image system—a full video monitoring system. This includes threeanalytical systems:

i) an AI image recognition system to analyze and determine the status ofthe human “object” (e.g., sitting, standing, reading, fighting, fallen,crawling on the floor, etc.);

ii) a human monitoring group to determine the status of the monitoredperson;

iii) a movement detection system to detect movement of the monitoredperson;

E) Both Thermal and Image (e.g., video, radar, etc.) system—This systemmay be used when a client desires a system with both privatecircumstance like a bathroom and regular open area like a living room.

An embodiment may include fewer or more of components, and variouscombinations of components. The following describes example embodimentsof these categories:

A) Geo system—detects person's location or movements within theirenvironment, and if they try to leave a defined area.

A sensing device is attached to the monitored person. In the primarymonitored location (property/residence), a different receiver will beinstalled to detect the person's location. This information will berecorded with a time stamp of how long they spent in each area, and sentto a central processing system where it is stored in the statusdatabase.

An “alert” notification will be sent when a person is in a dangeroussituation, which is determined based on algorithm settings customized bythe client. Various definitions of an alert/danger status may beemployed. The definition of an alert/danger status may be different fromsituation to situation and from person to person, but could includeconditions such as the person leaving the building or staying inbathroom for more than an hour.

Therefore, in an embodiment, the client/customer will be able to specifythe parameters of what constitutes a dangerous situation, and when thoseconditions are met, a central processing system will send out an alertmessage to the necessary parties in any digital format (voice mail,phone text messages, email, SMS, IM, etc.) through any network (e.g.,mobile, WAN, LAN, etc.). The scenario of when (and how long) to send thealert may be purely determined by the client. Applications running onmobile or other devices (e.g., apps running on a smart phone, smartappliances, IoT devices, etc.), may be employed to configure the systemparameters, to provide sensor and activity data (e.g., stove on,refrigerator door open, etc.), to receive and respond to alerts (e.g.,turning off an appliance, locking a door, turning off water, draining atub, etc.), etc., and various combinations thereof. In an embodiment,sensors may be built into various appliances and home features andfurnishings (e.g., walls, stoves, floors, bedding (e.g., mattresses,mattress covers), furniture (e.g., padding), etc., and may be configuredto detect any number of conditions, such as sleeping patterns, bodytemperature, wake-up time, accidents (e.g., falling out of bed), etc.

The sensing device could be anything attached to the body, such as animplant chip under the skin, a bracelet, a necklace, an electroniccoding device glued to fingernail or toenail, etc. In an embodiment,detection of a location of a monitored person may be performed usingother data, such as thermal imaging data from one or more thermalsensors, video imaging data from one or more image capturing devices;radar or lidar imaging data from one or more radar or lidar imagingsensors; sonar data from one or more sonar imaging devices; etc.

FIG. 1 shows an embodiment of a system 100 configured to monitor alocation of a person. As illustrated, the system 100 has a sensingdevice 102 which may be attached to the person and configured to sense alocation of the person or to send out signals from which the location ofthe person can be determined. As illustrated, the sensing device 102sends out a signal to one or more detectors 104, for example,periodically. The sensing device 102 may comprise one or more antennasfor communicating with the detectors 104. The detectors 104 receive thesignals from the sensor 102 and forward the signals to a transceiver 106which is communicatively coupled to a central processing system 108. Thecentral processing system 108 has one or more processors P, one or morememories 110 and one or more data bases, such as a database 112indicating a record status in the central processing system 108, and maybe configured to process the signals to determine the person's locationat various time periods and based on the person's location at thevarious time periods, determine whether there is a likelihood the personneeds assistance. The determinations may be based on other criteria anddata as well as location related data, such as a time of day, accrued orsnapshots of time in a particular location (e.g., too much time in thebathroom in a single time period, too much cumulative time in a bathroomin a window of time, too little time in the bathroom, too much timeelapsed between visits to a location such as the kitchen, etc.) day ofthe week, other sensor signals (such as signals indicative of a vitalsign, temperature, breathing rate, breathing depth, heart rate, bloodsugar level), etc. The various communication links may be wired orwireless. The central processing system 108 includes an artificialintelligence processor 114 configured to implement learning algorithmsas discussed in more detail elsewhere herein. The central processingsystem 108 also includes an interface 116 that may be employed toconfigure the system 100, for example by providing initial parameters,configuration information, override commands, etc., to the centralprocessing system 108. As illustrated, the system 100 comprises anetwork 118 configured to communicatively couple the system 100 to oneor more remote communication centers 120. The network 118 may beconfigured to provide secure communications.

FIG. 2 shows an embodiment of an environment 200 that may be employed tomonitoring and tracking as described herein. The environment 200includes a computing system 10. For example, the computing system 10 maybe configured as a tracking and monitoring system, a host server, suchas a security services server, a communications server, etc. Thecomputing system 10 may, for example, be operated by a service providingmonitoring services and related goods and services to a consumer, by aconsumer purchasing such goods or services from a service, by a vendor,such as a telecom service provider, an Internet service provider), etc.The computing system 10 may take the form of any of the variety of typesdiscussed above, which may run a networking client, for example aserver, a Web browser, etc. The computing system 10 comprises aprocessor unit 12, a system memory 14 and a system bus 16 that couplesvarious system components including the system memory 14 to theprocessing unit 12. The processing unit 12 may be any logical processingunit, such as one or more central processing units (CPUs), digitalsignal processors (DSPs), application-specific integrated circuits(ASIC), state machine, fuzzy-logic modules, etc., and variouscombinations thereof. Unless described otherwise, the construction andoperation of the various blocks shown in FIG. 2 are of conventionaldesign. As a result, such blocks need not be described in further detailherein, as they will be understood by those skilled in the relevant art.

The system bus 16 can employ any known bus structures or architectures,including a memory bus with memory controller, a peripheral bus, and/ora local bus. The system memory 14 includes read-only memory (“ROM”) 18and random access memory (“RAM”) 20. A basic input/output system(“BIOS”) 22, which can form part of the ROM 18, contains basic routinesthat help transfer information between elements within the computingsystem 10, such as during startup.

The computing system 10 also includes one or more spinning mediamemories such as a hard disk drive 24 for reading from and writing to ahard disk 25, and an optical disk drive 26 and a magnetic disk drive 28for reading from and writing to removable optical disks 30 and magneticdisks 32, respectively. The optical disk 30 can be a CD-ROM, while themagnetic disk 32 can be a magnetic floppy disk or diskette. The harddisk drive 24, optical disk drive 26 and magnetic disk drive 28communicate with the processing unit 12 via the bus 16. The hard diskdrive 24, optical disk drive 26 and magnetic disk drive 28 may includeinterfaces or controllers coupled between such drives and the bus 16, asis known by those skilled in the relevant art, for example via an IDE(Integrated Drive Electronics) interface. The drives 24, 26 and 28, andtheir associated computer-readable media, provide nonvolatile storage ofcomputer-readable instructions, data structures, program modules andother data for the computing system 10. Although the depicted computingsystem 10 employs hard disk 25, optical disk 30 and magnetic disk 32,those skilled in the relevant art will appreciate that other types ofspinning media memory computer-readable media may be employed, such as,digital video disks (DVD), Bernoulli cartridges, etc. Those skilled inthe relevant art will also appreciate that other types ofcomputer-readable media that can store data accessible by a computer maybe employed, for example, non-spinning media memories such as magneticcassettes, flash memory cards, USB sticks, solid state memories, RAMs,ROMs, smart cards, etc.

Program modules can be stored in the system memory 14, such as anoperating system 34 (for example, Windows, Android, Mac OS, 10S, etc),one or more application programs 36, other programs or modules 38, andprogram data 40. The system memory 14 also includes a server 41 forpermitting the computing system 10 to exchange data with sources such asWebsites of the Internet, corporate intranets, or other networks, aswell as other server applications on server computers. The server 41 maybe markup language based, such as hypertext markup language (HTML), andoperate with markup languages that use syntactically delimitedcharacters added to the data of a document to represent the structure ofthe document, etc.

While shown in FIG. 2 as being stored in the system memory 14, theoperating system 34, application programs 36, other program modules 38,program data 40 and server 41 can be stored on the hard disk 25 of thehard disk drive 24, the optical disk 30 and the optical disk drive 26and/or the magnetic disk 32 of the magnetic disk drive 28, a solid statememory, etc. A user can enter commands and information to the computingsystem 10 through input devices such as a keypad or keyboard 42 and apointing device such as a mouse 44. Other input devices can include amicrophone, joystick, game pad, scanner, touch screen, card reader, chipreader, etc. These and other input devices as illustrated are connectedto the processing unit 12 through an interface 46 such as a serial portinterface that couples to the bus 16, although other interfaces such asa parallel port, a game port or a universal serial bus (USB) can beused. A display or monitor 48 or other display devices may be coupled tothe bus 16 via video interface 50, such as a video adapter. Thecomputing system 10 can include other output devices such as speakers,printers, etc.

The computing system 10 can operate in a networked environment usinglogical connections to one or more repositories 6 and/or other computingsystems 8 a-8 n. The computer system 10 may employ any known means ofcommunications, such as through a local area network (LAN) 52 or a widearea network (WAN), a telecommunications network or the Internet 54.Such networking environments are well known and may include, forexample, any type of telecommunications network or other network, suchas CDMA, OFDMA, GSM, WiMAX, VoIP, WiFi, Internet Protocol, various IEEEstandard protocols, etc.

When used in a LAN networking environment, the computing system 10 maybe coupled to the LAN 52 through an adapter or network interface 56(communicatively linked to the bus 16). When used in a WAN networkingenvironment, the computing system 10 often includes a device, such as amodem 57, a mobile phone communication module or other device forestablishing communications over the WAN/Internet 54. As illustrated, amodem 57 is shown in FIG. 2 as communicatively linked between theinterface 46 and the WAN/Internet/Telecommunications network 54. In anetworked environment, program modules, application programs, or data,or portions thereof, can be stored in a server computer (for example,another configured computing system similar to the computing system 10).Those skilled in the relevant art will readily recognize that thenetwork connections shown in FIG. 2 are only some examples ofestablishing communication links between computers and/or other systemsand devices 60, and other links may be used, including wireless links.The devices may include, for example, sensors and monitors (see FIGS. 1,3, 5, 7 and 9).

The computing system 10 may include one or more interfaces such as slot58 to allow the addition of devices either internally or externally tothe computing system 10. For example, suitable interfaces may includeISA (Industry Standard Architecture), IDE, PCI (Personal ComputerInterface) and/or AGP (Advance Graphics Processor) slot connectors foroption cards, serial and/or parallel ports, USB ports (Universal SerialBus), audio input/output (I/O) and MIDI/joystick connectors, slots formemory, credit card readers, scanners, bar code readers, RFID readers,etc., collectively referenced as 60.

The term computer-readable medium as used herein refers to any mediumthat participates in providing instructions to processor unit 12 forexecution. Such a medium may take many forms, including but not limitedto, non-volatile media, and volatile media. Non-volatile media includes,for example, hard, optical or magnetic disks 25, 30, 32, respectively.Volatile media includes dynamic memory, such as system memory 14.

Common forms of computer-readable media include, for example, a floppydisk, a flexible disk, hard disk, magnetic tape, or any other magneticmedium, a CD-ROM, any other optical medium, punch cards, paper tape, anyother physical medium with patterns of holes, a RAM, a PROM, and EPROM,a FLASH-EPROM, any other memory chip or cartridge, as describedhereinafter, or any other medium from which a computer can read.

Various forms of computer readable media may be involved in carrying oneor more sequences of one or more instructions to processor unit 12 forexecution. For example, the instructions may initially be carried on amagnetic disk of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem 57 local to computer system 10 canreceive the data on the telephone line and use an infrared transmitterto convert the data to an infrared signal. An infrared detector coupledto the system bus 16 can receive the data carried in the infrared signaland place the data on system bus 16. The system bus 16 carries the datato system memory 14, from which processor unit 12 retrieves and executesthe instructions. The instructions received by system memory 14 mayoptionally be stored on storage device either before or after executionby processor unit 12.

The repository 6 is a permanent storage medium for data. The repository6 may be specific to each end user, or shared between some or all endusers. For example, different services vendors or concerned parties (forexample, family members or health care providers) may have separaterepositories or may share repositories. The repository 6 (only oneillustrated) may run on the same computing system as an applicationaccessing the repository, or on another computing system accessible overthe network 52, 54, or on a network of distributed repositories.

Embodiments of the computing system 10 of FIG. 2 may not include all ofthe illustrated components of the computing system 10, may containadditional components not shown in FIG. 2, and may not be configured asshown in FIG. 2. For example, a computing system 10 configured as homemonitoring system (see FIG. 1), may not include an optical disk driveand may include an application specific integrated circuit or a digitalsignal processor (not shown) to perform one or more of the functions ofthe home monitoring system. In another example, a detector ortransceiver may include one or more telecommunications modules to handlecall processing, such as CDMA, OFDMA, GSM, etc., call processing.

FIG. 3 is a flow diagram of an embodiment of a method 300 of monitoringa person's location, for example to determine whether there is anindication the person may need assistance based on information relatedto the person's location. As illustrated, the method 300 determineswhether there is an indication the person is safe or needs assistance.The method starts at block 302. At block 304, the method 300 gatherslocation related information and proceeds at block 306 to determinebased on the gathered information whether there is an indication theperson needs assistance. The determination may be based on the absenceof information, the change or lack of change of the information, otherinformation (for example, vital signs, information about the person'sphysical orientation, time of day, movement history), etc. Artificialintelligence algorithms may be employed to maintain (e.g., update,adjust, etc.) the criteria for determining whether the person is safe.When it is not determined that there is an indication the person needsassistance (as illustrated, when it is determined the person is safe),the method 300 proceeds to block 308 to update status information in adatabase, for example, to indicate that at a particular time the personis in a particular location. The method 300 proceeds from block 308 toblock 304.

When it is determined at block 306 that there is an indication theperson may need assistance (as illustrated, a determination the personis unsafe), the method 300 proceeds block 310 to take appropriate actionin response, such as generating and transmitting alert notices toconcerned parties (e.g., family and friends via designated devices),generating signals to initiate other actions (e.g., turning on lights,turning off a stove, etc.). The alert notices may provide real-timecondition status levels and summary information (e.g., “Warning!Abnormal Status: living room, no movement, 24 minutes, sitting posture”;“Normal Status: living room, sitting posture”; “Dangerous!: kitchen, nomovement, 18 minutes, lying posture”; etc.). The method 300 thenproceeds to block 308 to update status information, as discussed above.The method 300 proceeds from block 308 to block 304. The embodiment ofthe system 100 of FIG. 1 and the embodiment of the system 200 of FIG. 2may be configured to perform all or part of the method 300. Othersystems (e.g., the system 400 of FIG. 4) and various combinations ofsystems or components thereof, may be employed to implement all or partof the method 300 of FIG. 3.

FIG. 4 shows an embodiment of a system 400 configured to monitor ahealth status of a person. In an embodiment, a health tracking system400 monitors an overall health condition by tracking a range of person'svital signs. For example, a wrist-strap sensor 402 keeps track ofheart/breathing rate, blood oxygen, temperature, blood pressure, etc.Data is sent to the central processing system 408, and alerts aregenerated if any readings change dramatically or enter a “danger zone,”or are inconsistent with past behavior patterns. For example, bodytemperatures or changes in body temperature or characteristic bodytemperature histograms or other histograms (e.g., movement histograms,combined histograms) may indicate the existence of a medical emergency,such as a diabetic coma. For another example, heart/breathing rate ismonitored using artificial intelligence (AI) and remote sensingtechnologies. FIG. 14 is an illustration of monitoring breathing rateusing remote sensing technology. Changes in breathing rate orcharacteristic breathing rate histograms may indicate the existence of amedical emergency. Another example is collecting data using differentsensors during sleep in bed, like sweeting, heart rate, respiration,body temperature, sleeping cycle (REM (rapid eyes movement) sleep, NREM(non-rapid eyes movement) sleep, deep sleep, light sleep, etc.), breathrate, blood pressure. It should be understood that any sensor that iscapable of collecting body characteristic data can be used in the methodand system of the present disclosure. For example, radar systems may beconfigured to sense respiration and heart rates, in addition to orinstead of movement and position information. In another example, senseddata and data histories may be combined to determine whether a medicalemergency is indicated. For example, body and location positioninformation may indicate a monitored person is taking a bath. In such acircumstance, a temperature inconsistent with taking a bath, a durationlonger than an expected duration, etc., may generate a danger situation.In another example, a person who normally takes a three hour nap in theafternoon on the couch might trigger an alert when sensed data indicatesa nap is lasting a threshold period of time longer than a typical nap,when the sensed data indicates an unexpected temperature or temperaturechange, etc. An “alert” status will be sent out when it is determined aperson is in a “danger” situation. As illustrated, the system 400 has asensing device 402 which may be attached to the person and configured tosense one or more indications of a person's health, such as one or morevital signs. As illustrated, the sensing device 402 sends out a signalto one or more detectors 404, for example, periodically. The detectors404 receive the signals from the sensor 402 and forward the signals to atransceiver 406 which is communicatively coupled to a central processingsystem 408. The central processing system 408 has one or more processorsP, one or more memories 410 and one or more data bases 412, and may beconfigured to process the signals to determine an indication of theperson's health status at various time periods and based on signalsreceived regarding the person's health, such as indicators of a person'sheart/breathing rate, blood oxygen, temperature, blood pressure, bloodsugar levels, etc. This determination may be used to determine whetherthere is an indication the person needs assistance. The determinationsmay be based on other criteria and data, such as information andcriteria discussed above with respect to FIG. 1, active requests forassistance (e.g., pressing a help button on a pendant or mobile phone,calling a number, making a gesture indicative of a request for help,verbally requesting assistance), incomplete attempts to requestassistance (e.g., reaching for a phone, partially completing a gestureindicative of a request for assistance). Other sensors, such as locationsensors, may be employed in some embodiments. The system 400 may employan artificial intelligence (AI) processing module 414 to analyzereceived data and stored records to determine whether an alert statusshould be indicated. The system 400 has a configuration interface 416configured to provide configuration information to the system 400.

FIG. 5 is a flow diagram of an embodiment of a method 500 of monitoringindications of a person's health status, for example to determinewhether there is an indication the person may need assistance based oninformation related to the person's health status. As illustrated, themethod 500 determines whether there is an indication the person is safeor unsafe. The method 500 starts at block 502 and proceeds to block 504.At block 504, the method 500 gathers health related information andproceeds at block 506 to determine based on the gathered informationwhether there is an indication the person needs assistance. Thedetermination may be based on the absence of information, the change orlack of change of the information, other information (for example,location information, information about the person's physicalorientation, time of day), etc. When it is not determined that there isan indication the person needs assistance (as illustrated, when it isdetermined the person is safe), the method 500 proceeds to update statusinformation in a database at block 508, for example, to indicate that ata particular time the person is in a particular location and has aparticular physical orientation.

When it is determined at block 506 that there is an indication theperson may need assistance (as illustrated, a determination the personis unsafe), the method 500 proceeds to generate and transmit alertnotices to concerned parties (e.g., family and friends via designateddevices) and/or to take other actions as appropriate (e.g., turn onlights, turn off a stove, turn off water, lock or unlock a door, open orclose a door or window, etc). The method 500 then proceeds to block 508to update status information, as discussed above. The method 500proceeds from block 508 to block 504.

The embodiment of the system 100 of FIG. 1, the embodiment of the system200 of FIG. 2, and the embodiment of the system 400 of FIG. 4, may beconfigured to perform all or part of the method 500. Other systems(e.g., the system 600 of FIG. 6), and various combinations of systems orcomponents thereof may be employed to implement all or part of themethod 500 of FIG. 5.

FIG. 6 shows an embodiment of a system 600 configured to monitor whethera person needs assistance based on thermal imaging. In an embodiment, athermal image monitoring system 600 employs one or more infrared sensors602. For example, a number of strategically placed thermal cameras maybe installed on a property. Recorded status may be sent to the centralprocessing system database 612, and any status changes may be recordedwith time stamps. In an embodiment no personal identification may bestored in this system, ensuring that the privacy of the person will beprotected. An “alert” status will be sent out when it is determinedthere is an indication the person is in a “danger” situation. Forexample, it may be determined whether there is an indication the personis in danger using various analysis systems immediately after imageshave been recorded as follows:

i) An AI image recognition system 614 may be configured to analyze anddetermine the status of the human “object” (sitting, standing, reading,fighting, fallen, crawling on the floor, not moving, etc.). An alertstatus may be sent when the algorithms detect that the person could bein a “danger” situation.

ii) A human monitoring analysis person (e.g., a doctor, nurse) or group(e.g., nurses, nurse aides) may monitor the images, and members may noteany changes in the status of the images. The group may also determinewhether the conditions constitute a “danger” situation, in which case analert may be sent out. In an embodiment, permission may be required forhuman monitoring. In some embodiments, data in addition to or instead ofimages may be made available for remote monitoring. Such approaches maybe less invasive that summing assistance to the monitored individual'sresidence.

iii) A combined approach, for example, an automated system that sendsalerts to a human monitoring analysis group. For example, the monitoringsystem may also be configured to flag particular images or imagesequences for review by the monitoring analysis group.

As illustrated, the thermal sensing device(s) 602 send out a signal toone or more transceivers 606, for example, periodically. The transceiver606 is communicatively coupled to a central processing system 608. Thecentral processing system has one or more processors P, one or morememories 610 and one or more data bases 612, and may be configured toprocess the signals to determine an indication of the person'sorientation at various time periods and based on signals or imagesreceived related to the person's orientation (e.g., sitting, standing,reading, fighting, sleeping, crawling, bathing, etc.). Thisdetermination may be used to determine whether there is an indicationthe person needs assistance. The determinations may be based on othercriteria and data, such as information and criteria discussed above withrespect to FIG. 1. Other sensors, such as location sensors, healthstatus sensors, other orientation sensors, may be employed in someembodiments. It is noted that the thermal image sensors may providelocation sensing and health status information (such as temperature andbreathing rates, etc.) without being physically coupled to the personwhose status is being monitored. The system 600 comprises aconfiguration interface 616 (e.g., a keyboard, a Bluetooth receiver,etc.) to facilitate configuration of the system 600. Distributed systemsmay be employed (e.g., web-based AI servers).

FIG. 7 is a flow diagram of an embodiment of a method 700 of monitoringindications of a person's safety based on thermal images, for example todetermine whether there is an indication the person may need assistancebased on information related to the person's location, physicalorientation, health status, etc. As illustrated, the method 700determines whether there is an indication the person is safe. The methodstarts at block 702 and proceeds to block 704 to gather thermal imagesand extracts information therefrom and proceeds to determine at block706 based on the gathered images and extracted information whether thereis an indication the person needs assistance. The determination may bebased on the absence of information, the change or lack of change of theinformation, other information (for example, other location information,other health-status information, time of day), etc. When it is notdetermined that there is an indication the person needs assistance (asillustrated, when it is determined the person is safe), the method 700proceeds to block 708 to update status information in a database, forexample, to indicate that at a particular time the person is in aparticular location, has a particular orientation, etc.

When it is determined at block 706 that there is an indication theperson may need assistance (as illustrated, a determination the personis unsafe), the method 700 proceeds at block 710 to generate andtransmit alert notices to concerned parties (e.g., family and friendsvia designated devices) and/or to take other action as appropriate(e.g., asking the person if they are alright before triggering analarm). The method 700 then proceeds to block 708 update statusinformation, as discussed above. The method 700 proceeds from block 708to block 704.

The embodiment of the system 600 of FIG. 6, the embodiment of the system200 of FIG. 2, and the embodiment of the system 400 of FIG. 4 may beconfigured to perform all or part of the method 700. Other systems(e.g., the system 100 of FIG. 1, etc.), and various combinations ofsystems or components thereof may be employed to implement all or partof the method 700 of FIG. 7.

FIG. 8 shows an embodiment of a system 800 configured to monitor whethera person needs assistance based on video imaging. In an embodiment, avideo image monitoring system 800 employs one or more video cameras 802.For example, a number of strategically placed video cameras may beinstalled on a property. Recorded status may be sent to a centralprocessing system database 812, and any status changes may be recordedwith time stamps. An “alert” status may be sent out when it isdetermined there is an indication the person is in a “danger” situation.For example, it may be determined whether there is an indication theperson is in danger using various analysis systems immediately afterimages have been recorded as follows:

i) An AI image recognition system 814 configured to analyze anddetermine the status of the human “object” (sitting, standing, reading,fighting, fallen, crawling on the floor, etc.). An alert status may besent when algorithms of the AI system 814 detect that the person couldbe in a “danger” situation.

ii) A human monitoring analysis group (e.g., nurses) may monitor theimages, and members of the group may note any changes in the images. Thegroup may also determine whether the conditions constitute a “danger”situation, in which case an alert may be sent out.

iii) A combined approach, for example, an automated system that sendsalerts to a human monitoring analysis group. This might be desirable toreduce false positive alerts or to lower the threshold for an automaticalert, etc. The automated system may store responses determined by thegroup and employ the information to update the algorithms (e.g., toreduce false triggers by the automated system).

As illustrated, the video recording device(s) 802 send out video signalsto one or more transceivers 806, for example, periodically. Thetransceiver 806 is communicatively coupled to a central processingsystem 808. The central processing system has one or more processors P,one or more memories 810 and one or more data bases 812, and may beconfigured to process the signals to determine an indication of theperson's orientation at various time periods and based on signalsreceived regarding the person's orientation (e.g., sitting, standing,reading, fighting, sleeping, crawling, bathing, etc.). Thisdetermination may be used to determine whether there is an indicationthe person needs assistance. The determinations may be based on othercriteria and data, such as information and criteria discussed above withrespect to FIG. 1. Other sensors, such as location sensors, healthstatus sensors, thermal sensors, and various combinations, may beemployed in some embodiments. It is noted that the video image sensors802 may provide location sensing and health status information (such asbreathing rates, etc.) without being physically coupled to the personwhose status is being monitored. The system 800 comprises aconfiguration interface 816 (e.g., a keyboard, a Bluetooth receiver,etc.) to facilitate configuration of the system 800.

FIG. 9 is a flow diagram of an embodiment of a method 900 of monitoringindications of a person's safety based on video images, for example todetermine whether there is an indication the person may need assistancebased on information related to the person's location, physicalorientation, health status, etc. As illustrated, the method 900determines whether there is an indication the person is safe. The methodstarts at block 902 and proceeds to block 904 to gather video images andextract information therefrom and proceeds to block 906 to determinebased on the gathered images and extracted information whether there isan indication the person needs assistance. The determination may bebased on the absence of information, the change or lack of change of theinformation, other information (for example, other location information,other health-status information, time of day), etc. When it is notdetermined at block 906 that there is an indication the person needsassistance (as illustrated, when it is determined the person is safe),the method 900 proceeds to block 908 to update status information in adatabase, for example, to indicate that at a particular time the personis in a particular location, has a particular breathing rate,temperature, etc.

When it is determined at block 906 that there is an indication theperson may need assistance (as illustrated, a determination the personis not safe), the method 900 proceeds to block 910 to generate andtransmit alert notices to concerned parties (e.g., family and friendsvia designated devices) and/or to take other action as appropriate(e.g., to request data from one or more sensor related to a conditionsuggesting the person might need assistance, etc.). The method 900 thenproceeds to update status information at block 908, as discussed above.The method 900 proceeds from block 908 to block 904.

The embodiment of the system 800 of FIG. 8, the embodiment of the system200 of FIG. 2, and the embodiment of the system 400 of FIG. 4 may beconfigured to perform all or part of the method 700. Other systems(e.g., the system 600 of FIG. 6, etc.), and various combinations ofsystems or components thereof may be employed to implement all or partof the method 900 of FIG. 9.

Embodiments of methods described or illustrated herein may containadditional acts not described or shown in the figures, may not containall of the acts described or shown in the figures, may perform actsdescribed or shown in various orders, and may be modified in variousrespects. For example, the determinations illustrated in FIGS. 1, 3, 5,7 and 9 may in some embodiments be made based on a combination of thevarious types of data gathered by the systems illustrated in FIGS. 1, 2,4, 6 and 8. For example, a monitoring system may employ a combination oflocation sensors coupled to a person, health sensors coupled to aperson, thermal sensors, video sensors, other sensors (such as windowand door sensors), etc., to receive indications that may be employed todetermine whether there is an indication that a person needs assistance.In another example, the process may be iterative. For example, one ormore of the described methods may respond to an indication that a personmay be in danger by gathering additional information before determiningto generate an alarm signal.

For example, FIG. 10 illustrates an embodiment of a monitoring system1000 employing both thermal image sensors 1002 a and video imagemonitors 1002 b. Such an embodiment may be employed, for example, whenit is desirable to monitor places such as bathrooms, where thermalsensors might be employed to increase privacy, as well as more publicplaces, such as living rooms and outside decks. Data from thermalsensors and video image monitors may be combined in some embodiments.

As illustrated, the monitors 1002 a, 1002 b send out signals to one ormore transceivers 1006, for example, periodically. The transceiver 1006is communicatively coupled to a central processing system 1008. Thecentral processing system has one or more processors P, one or morememories 1010 and one or more data bases 1012, and may be configured toprocess the signals to determine an indication of the person'sconditions at various time periods and based on signals receivedregarding the person's conditions (e.g., orientation conditions(sitting, standing, reading, fighting, sleeping, crawling, bathing,etc.); health status conditions, facial expressions, etc.). Thesedeterminations may be used to determine whether there is an indicationthe person needs assistance. The determinations may be based on othercriteria and data, such as information and criteria discussed above withrespect to FIG. 1. Other sensors, such as location sensors, healthstatus sensors, thermal sensors, and various combinations, may beemployed in some embodiments. The system 1000 comprises a configurationinterface 1016 (e.g., a keyboard, a Bluetooth receiver, etc.) tofacilitate configuration of the system 1000. Detectors (see detector 104of FIG. 1) may be employed in some embodiments.

At least some embodiments of a monitoring system may have full duplexcommunication ability where a monitored person (MP) can communicate witha) the computer system; and b) people who locate in a data communicationcenter (e.g., an ONSTAR® communication center), and vice versa.

In an embodiment, the monitoring system includes AI machine learningalgorithms (implemented, for example, by AI modules such as the AImodule 114 of FIG. 1) that may increasingly identify objects, humanphysical movement and their facial expressions to determine and/orverify whether the study subject (human) is at risk. The monitoringsystem may be configured to predict events by recognizing patterns ofmovement or behaviors that typically precede them. For example, beforecollapsing, many people will sway back and forth or side to side.Similarly, repeating walking patterns over and over could signalconfusion induced by some medical condition. On the other hand, for somepeople repeating walking patterns (e.g., pacing in a particular place)may be a normal occurrence.

In an embodiment, a threshold for triggering a “danger” red flag has 3levels which are based on statistical analysis of the data collectionfrom the normal behavior of the study human (MP). If a movement, livinghabit, facial expression or any other abnormal behavior appears (whichmay fall off a pre-set threshold percentile in the normal distributionrange of a bell curve—e.g. 68%—one standard deviation from the mean), analarm may be triggered and this statistical analysis may be captured tobe used in the future to improve accuracy as part of the AI machinelearning process.

AI and other data analysis may be employed which relies on one or moreof global predictions (e.g., predictions based on histograms associatedwith a generic population), demographic based predictions (e.g.,predictions based on histograms associated with a specific demographic(e.g., the population of a particular city or region, people of aparticular race or age or other risk factor, combinations thereof)), andindividual based predictions (e.g., predictions based on histogramsassociated with a specific individual (e.g., a person's history ofmovement), and various combinations thereof.

In an embodiment, a monitoring computer system may also accomplishnormal daily tasks with voice commands to control household electronicdevices like:

-   -   Setting an alarm clock,    -   Reminders for daily activities like taking pills, cooking times,        setting calendar event reminders, etc.    -   Making phone calls using the full duplex system simply by        asking: “Amina (referring to an example brand name for a        monitoring system), please call my son” (or some similar phrase        to activate a phone call).    -   Turn on/off and make adjustment to household devices (e.g. TV,        radio control, audio devices, volume control, channel switching,        open/close or adjust the blinds/curtains, etc.)

In an embodiment, the voice commands may be used to trigger an alert. Inan embodiment, the system may prompt a user to respond. No response, aninappropriate response, or a response which does not correspond to anexpected response may prompt an alert. In an embodiment, voicerecognition subsystems may be employed, for example, to identify voicecommands, to authenticate voice commands, to identify the person to bemonitored (e.g. to trigger a robot to move to better monitor theperson), to identify another person in a monitored location, etc., andvarious combinations thereof. In an embodiment, a push to talk systemmay be employed.

In an embodiment, a monitoring system may also be a secure alarm systemwhich can detect any invader/thief entering the property withoutpermission. AI algorithms may be employed to reduce false alarms for themonitoring system as well (e.g., time information, human recognitioninformation, pattern recognition, etc.)

In an embodiment, the components of a monitoring system (e.g., thesystems of FIGS. 1, 4, 6 and 8) might include Image capture devices(infrared, 3-D camera).

An embodiment might include AI machine learning systems (e.g., AIsystems 114, 414, 614 and 814 of FIGS. 1, 4, 6 and 8) where algorithmsin the system are configured to analyze and define a monitored person'sproperties and a monitored person's environment properties, identifywhether the monitored person is at risk, for example by using AI systemidentifiers as discussed below. The AI identifiers might comprise, forexample, Object identifiers (which define human, animal, and otherobjects—chair, table, sofa, etc.); Location identifiers (which define amonitored persons location—bedroom, bathroom, kitchen, etc.); Positionidentifiers (which define a position such as sitting, lying down,crawling, etc.); Time identifiers (which identify the time of the daywhen a monitored person is in each location); Sound identifiers (whichidentify sounds generated by human and non-human objects—e.g. Microwave,fire alarm, TV, etc.); Motion identifiers (which identify body movementcharacteristics); Physical Status identifier (which identify physicalhealth condition characteristics); Emotional identifiers (which, forexample, identify facial expression characteristics, other physicalcharacteristics indicative of emotion (e.g., position, temperature,smells, etc.)); Motion and Emotional identifiers (which, for example,identify both facial expression and body movement characteristics);etc., and various combinations thereof.

At least some embodiments are configured to communicate with a remotecommunication call center, which can directly communicate to themonitored person's household with a full duplex communication (two-way)ability.

At least some embodiments may comprise a secure network system wheredigital information (e.g. data record, images, etc.) can be transmittedto the computer servers, CPU, emergency communication center and anynodes/parties within the network. A secure full duplex voicecommunication can be transmitted to different nodes/parties within thenetwork.

At least some embodiments may be configured to provide reports and/oraccess to other devices (e.g. handheld devices, computers, etc.) whereauthorized interested parties (e.g. a monitored person's families andfriends) may access the recorded record of the MP proprieties (e.g.record of how long the MP slept last night; what is the average nap timein the afternoon, etc.).

In at least some embodiments, an AI system of a monitoring system (e.g.,AI systems 114, 414, 614, 814, 1014 of systems 100, 400, 600, 800, 1000of FIGS. 1, 4, 6, 8 and 10) may use one or more of the followingidentifiers to identify different properties of a monitored person and amonitored person's environment: Object identifiers (define human,animal, or other object—chair, table, etc.); Location identifiers(define the MP's location—bedroom, bathroom, kitchen, etc.) Positionidentifiers (sitting, lying down, crawling, etc.); Time identifiers(identify the time of the day when the MP is in each location); Soundidentifiers (Sound that is generated by human and non-human objects—e.g.Microwave, fire alarm, TV, etc); Motion identifiers (body movementcharacteristics); Emotional identifier (facial expressioncharacteristics); Physical status identifier (which identify healthcondition characteristics—e.g. heart rate, blood glucose content, coldsweat, etc); Motion and Emotional identifier (both facial expression andbody movement characteristics); etc.; and various combinations thereof.

Monitored person properties that are defined by the above identifiersmay be used to generate different measurements to compare to a thresholdto trigger an alarm. This collection of data is not limited to, butincludes the following properties: locations; pattern of facialmovement; pattern of body movement; hours of the day and night indifferent locations of the monitoring property; pattern of voice, tone,and any sound related properties; physical body condition; thecombination of two or more of the above properties; etc. An AI systemaccuracy may increase with the AI machine learning algorithms bycollecting more observation data (characteristics/properties collectionis defined by different identifiers—see above identifier's definition).With the continuously increasing the data set, the accuracy may increaseevery day with the AI ability of learning.

The collection data that improves the AI machine learning algorithms mayinclude: (1) a current monitored person's properties (e.g. body movementand facial activities); and (2) other studied human subject's propertiesthat are collected outside the current monitored person.

In an embodiment, different alarm levels and responses thereto may beemployed. For example, a Level 1 alarm condition may cause a monitoringsystem (e.g., system 100 of FIG. 1) to call a monitored person using afull duplex voice communication system when an abnormalbehavior/characteristic is detected. If the monitored person responds,“it's fine”, the system may learn this as new pattern property toincrease future recognition accuracy. If there is no response from theMP, a “level 2” alarm may be triggered. In an embodiment, a Level 2alarm may send signals to a monitoring data center (e.g., monitoringcenter 120 of FIG. 1) indicating real people at the monitoring datacenter should call the monitored person. In an embodiment, if themonitored person does not answer, a Level 3 alarm may be triggered. Inan embodiment, a Level 3 alarm may indicate the data center shouldfollow emergency steps to contact relatives/friends, fire fighters,medics, hospital/health care providers, etc. The monitored person/familymember(s) may provide this emergency list of contacts when signing upfor this monitoring system or configuring the monitoring system. In someembodiment, the Levels may trigger different responses, and additionallevels may be employed. For example, the monitoring system may send anautomated message based on emergency contact information beforecontacting a remote communication center under specified conditions.

The stages of the alarm level may be recorded and the AI machinelearning configured to learn when false alarms are being triggered. Thisnew pattern of abnormal properties/behaviors/characteristics whichtriggered the alarm may be stored and thus learned. The system may usethe stored patterns to improve the accuracy of predictions in the futureas part of the AI machine learning process.

In an embodiment, a prediction may be based on previously studiedstatistic analysis of patterns in properties (e.g. movement, facialexpression), and predict and define normal characteristics (e.g. facialexpression, body movement, hours of sleep, location of sleep—nap in sofaduring daytime vs. bedroom sleep at night). If the prediction fallsbeyond a threshold (e.g., a standard threshold such as 68.2%(34.1%+34.2%) line—the threshold number may be edited according to thesituation and the consensus between the MP/MP's family and theprogrammer) from a normal distribution curve, an alarm will betriggered. FIG. 11 shows an example of thresholds employed in a Bellcurve.

In an embodiment, an AI algorithm is configured to predict movement of amonitored person based on: (1) the current MP's properties (e.g. bodymovement and facial activities); and (2) other studied human subject'sproperties that are collected outside the current MP. When the recordedpriorities fall beyond a threshold percentile (e.g., 68%—one standarddeviation from the mean) of the predicted properties (e.g., movementsequence; staying in the bathroom longer than the standard threshold),an alarm may be triggered. In an embodiment, a threshold curve ofproperties measurement may change accordingly as the AI system continuesto learn, develop and improve the accuracy with the AI machine learningability. The AI system will learn, justify/adjust threshold(s), andtailor responses toward the living habits of an individual monitoredperson.

For example, if a Tai Chi series pattern of movements has never beenobserved and recorded, the alarm may be triggered as the MP's propertiesare beyond the normal threshold percentile. However, as more and morerecorded Tai Chi practice movement occurs, the system will learn andtreat this series of Tai Chi moments as the MP's normal pattern ofbehavior. This also may be applied to facial movement, and other typesof body movement and living behaviors (e.g., hours of sleep, left handedand right handed changes, body temperature changes, etc.).

In another example, an AI system may be configured to predict events byrecognizing patterns of movement or behaviors that typically precedethem—before collapsing, many people will sway back and forth or side toside, repeating walking patterns over and over could signal confusioninduced by some medical condition, and there may be othercharacteristics that happen just before strokes or heart attacks.

In addition, each property (e.g. location; time) may have differentthresholds of property percentile curve to trigger the alarm. Thethreshold of measurement may be different depending on a specificcombination of identified properties.

In another example, the hours (time identifier) of laying down (positionidentifier) in the kitchen may be five minutes, laying down in theliving room sofa (location identifier and object identifier) may be twohours, but the threshold of laying down in the bedroom may be six hoursat night (12 am to 6 am) and three hours during daytime (1 pm to 6 pm).In other words, the location and time would have their own standarddeviation threshold for any given time of the day and the location ofthe monitoring area.

In another example, a monitored person may normally stay in the bathroomfor 30 minutes (e.g. +/−3% time different). If the monitored personstays longer in the bathroom more than this time frame, the alarm may betriggered.

In another example, a “Physical Status identifier” may identify physicalhealth condition characteristics (e.g., a heart rate, blood glucosecontent, cold sweat condition, etc.). The physical characteristics mayhave their own pattern, or be including in the pattern comprising othercharacteristics, and a bell curve may be employed to determine athreshold by defining normal and abnormal body physical behavior basedindicators related to those characteristics. I

These types of property thresholds may also be edited according to thesituation and the consensus between the MP (Monitored Person)/MP'sfamily and the programmer (e.g. hours to sleep, where to sleep, etc.).

Embodiments may provide for flexible responses. For example, amonitoring person could choose the following when the alarm is beingtriggered: (1) send the full capture video to the call center or (2)choose not to send video to the call center. If a household has morethan one monitored person that needs to monitored, the patterned andmonitoring properties may be modified to fit the household condition.For example, it may be less focused on facial pattern and movement, butmore focus on laying down properties in location, durations or locationof stay in certain hours of the day (e.g. bathroom for less than 30minutes during day time). In another example, the AI may employidentifiers to identify the monitored person (e.g., based on size,activity patterns, facial recognition, etc.) and be configured toconsider these identifiers when determining whether a person is indanger and the appropriate response to such a determination.

In an embodiment, a robot with sensors (e.g., image, thermal, etc.) mayfollow an individual on a property instead of or in addition to usingfixed sensors. An AI system using a statistical record pattern mayemploy spot and zone based pattern learning. FIG. 12 illustrates examplezone and spot based temperature curves. In an embodiment, if a newbehavior is detected (e.g., a behavior or condition inconsistent withpast behavior which may indicated a person needs assistance), the systemmay prompt a monitored person (or a monitoring human) to confirm thebehavior or condition does not indicate assistance is needed. If it isconfirmed that the behavior or condition does not indicate assistance isneeded, the system may update histogram data or other data to reflectthat this type of behavior does not present a threat. If no response isreceived or a response is received indicating assistance is needed, thesystem may generate an alert. For example, a person may decide to employa new exercise routine (e.g., yoga, aerobics, etc.) which the system maydetect as a new behavior with associated data (e.g., movement periods,heart rates, temperatures, position information, etc). The system mayprompt the monitored person or a monitoring person to confirm adangerous situation is not presented. The system will learn that the newbehavior does not present a danger (within certain parameters). In anembodiment, the robot or robots may perform other functions, e.g,vacuuming, playing music, controlling other appliances, connecting tothe internet, placing phone calls, etc.). In an embodiment, a robot orother system component may be configured to respond to voice commands.In an embodiment, the robot may be able to provide assistance to themonitored person either in the normal course (e.g., carrying things forthe person, assisting the person in getting up, pulling the person up,etc.), or in response to detection of a dangerous condition (e.g.,assisting the person in getting up, pulling the person up, in additionto or instead of generating an alarm).

Global and zone or area based criteria may be employed. For example,monitoring in a work-out area and monitoring in a sleeping area may bothemploy global thresholds and histograms, while also using zone basedthresholds and histograms. For example, the monitoring of the work-outarea may also employ work-out area histograms and thresholds in additionto the global thresholds and conditions.

Various techniques and systems may be employed by the AI to interpretthe sensor data and histograms. For example, artificial neural networksand various programs and systems (e.g., Tensile Flow, On Premise) may beemployed. Sparse dictionaries and dictionary transforms may be employed.Support vector machines may be employed, etc., and various combinationsthereof.

In some embodiments, various lenses may be employed, e.g., a wide angle15 mm lens, a 70-90 degree fish eye, infrared lenses, etc.

In some embodiments, a monitored individual may be reminded to orprompted to perform certain activities. For example, if a monitoredperson has been sitting for a threshold period of time, the person maybe prompted to walk, stand up or stretch. If a monitored person issupposed to take medicine and activity consistent with the taking of themedicine is not detected, the person may be prompted to take themedicine or to confirm the medicine has been taken, etc. The thresholdmay depend on the position of the person, the location of the person,the time of day, etc. For example, lying down in a bedroom for a certainnumber of hours may not generate a prompt or alert from the system if itis consistent with a normal sleeping pattern of the person. On the otherhand, lying down in the kitchen may generate a prompt or alert after afew minutes or seconds. The time may depend on other factors (e.g.,pulse, temperature, etc.).

Some embodiments may prompt a monitored person to eat or trigger analarm if eating activity has not been detected for a threshold period oftime. The threshold(s) may vary based on the response (e.g., a thresholdfor prompting may be shorter than a threshold for triggering an alert),global data (e.g., for most people), demographic data (e.g., diabetics),and specific data (e.g., the monitored person always eats at 9:00 a.m.on weekdays). Some embodiments may make predictions based on multiplefactors, e.g., histogram data suggesting a low-blood sugar level, suchas body temperature, position and the time since eating activity wasobserved.

Some embodiments may monitor a person for signs of depression (less bodymovement or different positions and times of body movement, failures toturn on lights consistent with prior behavior, etc.

FIG. 13 illustrate an embodiment of a system 1300 and a method 1350 ofoperating such a system. While method 1350 is described with referenceto system 1300, other systems, such as the various other systemsdescribed herein, may be employed to monitor an individual and detectmotion. In an embodiment, three images im0, im1 and im2 are captured andconverted to a normalized grayscale. The pixels of the images are thenpassed through the function (img1-img0) and (img1-img2), and normalizedagain. The resulting image is a pixel-by-pixel map of motion during thecapture time of the three images. In an embodiment, detection of amaximum time without movement may generate a value Y indicative of atime since movement occurred based on a classifier prediction. Theclassifier makes a prediction and reports the class, image name and timeto a database. A program queries the data base to make certain the Yvalues do not exceed threshold Y values, which may be associated withthe type of activity (e.g., sleeping, eating, watching TV, etc.). In anembodiment, the classifications are generated and passed to a motiondetection and threshold analysis before being reported to a database(which may be remote, e.g., in a cloud). Local alerts are triggered. Inan embodiment, both classification and trigger schemes may be employed(e.g., local evaluation followed by cloud evaluation, either of whichmay trigger a response (e.g., query to the monitored person, alert,etc.).

In FIG. 13, a system 1300 and a method 1350 are illustrated. Timeinformation is provided to a classification timer 1302 and an imagecapture device 1304. The image capture device 1304, which as illustratedis a camera, captures images of an environment in which an individual isbeing monitored. The images are provided to classifier 1306, whichclassifies the images and provides the images to the classificationtimer 1302. The classification timer 1302 provides or retrieves classand time information (e.g., thresholds) associated with variousdangerous conditions to/from a relational database management system1308. Based on comparisons of the class and time information associatedwith images of an image stream to the information in the relationaldatabase system, an alert may be generated at 1310.

In method 1350, at 1352 the system 1300 checks whether an image of animage stream is associated with a new class, e.g., has the individualmoved from a sitting position to a standing position. When it is notdetermined at 1352 that the image is associated with a new class, themethod 1350 proceeds from 1352 to 1354, where it is determined whetherthe current time since the last movement is exceeded for theclassification. When it is determined at 1352 that an image isassociated with a new class, the method 1350 proceeds to 1356, where acurrent position timer is reset. The method 1350 proceeds from 1356 to1358 and 1360.

When it is determined at 1354 that the current time since the lastmovement is exceeded for the classification, the method proceeds to1362, where an alert trigger is generated. As illustrated, the alerttrigger is provided to an alert system, and may contain informationrelated to the classification and timing that triggered the alerttrigger, which may be used by the alert system to determine anappropriate response. The method proceeds from 1362 to 1358.

When it is not determined at 1354 that the current time since the lastmovement is exceeded for the classification, the method proceeds to 1358and 1360, where an alert trigger is generated. As illustrated, the alerttrigger is provided to an alert system, and may contain informationrelated to the classification and timing that triggered the alerttrigger, which may be used by the alert system to determine anappropriate response. The method proceeds from 1362 to 1358.

At 1358, information related to the image of the image stream, such astiming and classification information, is provided to the relationaldatabase management system, which may use the data to train an AI systemas discussed in more detail elsewhere herein. At 1360, the method 1350waits for new data. When new data is received, the method proceeds from1360 to 1352.

Embodiments of the method 1352 may perform acts in various orders, maycombine acts, may divide acts into separate acts, may contain fewer ormore acts than illustrated, may perform acts in parallel or in variousorders, and various combinations thereof.

An embodiment may employ thermal data specific to regions of a monitoredperson, specific activities, etc. For example, the head of an individualmay be identified (it is often the hottest part of a human) and datarecorded and used to classify a situation as potentially hazardous. Thetable below shows an example database. As shown in the example, layingin the kitchen for 5 minutes with a temperature of 39 degrees C.triggers an alert.

Position Last Stay Body temperature Time Location (posture) move still(Celsius) 13:57 Living room stand 13:58 1 min  37 14:05 Kitchen stand13:59 3 mins 37 14:10 Living room sit 14:06 4 mins 38 14:20 Kitchen lay14:11 5 mins 39

Some embodiments may monitor other environments and types of subjects,such as children in a school room or school yard, animals (e.g., dogs,pigs, cows, etc.) to identify abnormal behavior and/or dangeroussituations (e.g., excessive heat, poor air quality), etc.

FIG. 15 illustrates an embodiment of a monitoring system 1500 employinga plurality of sensors 1502, as illustrated the plurality of sensorsinclude thermal image sensors 1502 a and radar images sensors 1502 b.Such an embodiment may be employed, for example, when it is desirable tomonitor places such as bathrooms, where thermal sensors or radar imagesensors might be employed to increase privacy, as well as more publicplaces, such as living rooms and outside decks. Data from thermalsensors, radar sensors and other sensors (e.g., lidar sensors, videoimage monitors, etc.) may be combined in some embodiments.

As illustrated, the sensors 1502 a, 1502 b send out signals to one ormore transceivers 1506, for example, periodically. The transceiver 1506is communicatively coupled to a central processing system 1508. Thecentral processing system has one or more processors P, one or morememories 1510 and one or more data bases 1512, and may be configured toprocess the signals to determine an indication of the person'sconditions at various time periods and based on signals receivedregarding the person's conditions (e.g., orientation conditions(sitting, standing, reading, fighting, sleeping, crawling, bathing,etc.); health status conditions, facial expressions; movement conditions(e.g., speed, direction, trajectory, potential and actually collisions,etc.); etc.) These determinations may be used to determine whether thereis an indication the person needs assistance. The determinations may bebased on other criteria and data, such as information and criteriadiscussed above with respect to FIG. 1. Other sensors, such as locationsensors, health status sensors, pendant sensors or devices, and variouscombinations, may be employed in some embodiments. The system 1500comprises a configuration interface 1516 (e.g., a keyboard, a Bluetoothreceiver, etc.) to facilitate configuration of the system 1500.Detectors (see detector 104 of FIG. 1) may be employed in someembodiments.

Some embodiments may take the form of or comprise computer programproducts. For example, according to one embodiment there is provided acomputer readable medium comprising a computer program adapted toperform one or more of the methods or functions described above. Themedium may be a physical storage medium such as for example a Read OnlyMemory (ROM) chip, or a disk such as a Digital Versatile Disk (DVD-ROM),Compact Disk (CD-ROM), a hard disk, a memory, a network, or a portablemedia article to be read by an appropriate drive or via an appropriateconnection, including as encoded in one or more barcodes or otherrelated codes stored on one or more such computer-readable mediums andbeing readable by an appropriate reader device.

Furthermore, in some embodiments, some or all of the methods and/orfunctionality may be implemented or provided in other manners, such asat least partially in firmware and/or hardware, including, but notlimited to, one or more application-specific integrated circuits(ASICs), digital signal processors, discrete circuitry, logic gates,state machines, standard integrated circuits, controllers (e.g., byexecuting appropriate instructions, and including microcontrollersand/or embedded controllers), field-programmable gate arrays (FPGAs),complex programmable logic devices (CPLDs), etc., as well as devicesthat employ RFID technology, and various combinations thereof. Forexample, embodiments of a home monitoring system may be implemented asdiscussed above (e.g., partially in hardware, partially with controllersexecuting instructions, etc.).

The various embodiments described above can be combined to providefurther embodiments. Aspects of the embodiments can be modified, ifnecessary to employ concepts of the various patents, applications andpublications to provide yet further embodiments.

These and other changes can be made to the embodiments in light of theabove-detailed description. In general, in the following claims, theterms used should not be construed to limit the claims to the specificembodiments disclosed in the specification and the claims, but should beconstrued to include all possible embodiments along with the full scopeof equivalents to which such claims are entitled. Accordingly, theclaims are not limited by the disclosure.

1. A safety monitoring system, comprising: a plurality of sensorsconfigured to generate signals indicative of a current condition relatedto a safety status of a person; and one or more processing devicescoupled to the plurality of sensors and configured to: determine asafety status of the person based on the signals indicative of thecurrent condition and stored information related to the safety status ofthe person; update the stored information related to the safety statusof the person based on the signals indicative of the current condition;and in response to the determined safety status indicating the personmay be in danger: initiate one or more actions based on the determinedsafety status; monitor responses to the one or more initiated actions;and update the stored information related to the safety status of theperson based on the monitored responses.
 2. The safety monitoring systemof claim 1 wherein the one or more initiated actions comprise:generating an alert signal to the person; transmitting a signal to aremote server; generating an alert message based on stored contactinformation; or combinations thereof.
 3. The safety monitoring system ofclaim 1 wherein the plurality of sensors include: a location sensor; athermal sensor; a health status monitoring device; a radar sensor; avideo capturing device; or combinations thereof.
 4. The safetymonitoring system of claim 1 comprising a device configured to performvoice communications.
 5. The safety monitoring system of claim 1 whereinthe stored information comprises property identifiers indicative ofconditions and the one or more processing devices comprise an artificialintelligence module configured to compare the signals indicative of thecurrent condition to the stored property identifiers and to determinethe safety status of the person based on the comparison.
 6. The safetymonitoring system of claim 5 wherein the determining the safety statuscomprises determining a position of a property identifier on acharacteristic property-identifier curve based on the signals indicativeof the current condition.
 7. The safety monitoring system of claim 6wherein the characteristic property identifier curve is a Bell curvebased on stored property identifiers indicative of the characteristic.8. The safety monitoring system of claim 6 wherein the characteristicproperty-identifier curve is related to one or more of: an objectidentifier; a location identifier; a position identifier; a timeidentifier; a sound identifier; a motion identifier; a physical statusidentifier; or an emotional identifier.
 9. The safety monitoring systemof claim 6 wherein the determining the safety status comprisesdetermining whether the position is within one or more thresholddeviations from a mean of the Bell curve.
 10. The safety monitoringsystem of claim 1 wherein the plurality of sensors include at least oneof: an infrared camera; and a 3-D camera.
 11. The safety monitoringsystem of claim 1 wherein the stored information includes an initialdata set and updating the stored information comprises adding data tothe initial data set.
 12. A method, comprising: receiving signalsindicative of a current condition related to a safety status of aperson; determining, using at least one processing device, a safetystatus of the person based on the signals indicative of the currentcondition and stored information related to the safety status of theperson; updating, using the at least one processing device, the storedinformation related to the safety status of the person based on thesignals indicative of the current condition; and when the determinedsafety status indicates the person may be in danger: initiating one ormore actions based on the determined safety status; monitoring responsesto the one or more initiated actions; and updating the storedinformation related to the safety status of the person based on themonitored responses.
 13. The method of claim 12 wherein the one or moreinitiated actions comprise one or more of: generating an alert signal tothe person; transmitting a signal to a remote server; or generating analert message based on stored contact information.
 14. The method ofclaim 12 wherein the signals indicative of the current conditioninclude: a signal indicative of a location of the person; a signalindicative of a temperature; a signal indicative of a health status ofthe person; an imaging signal; or combinations thereof.
 15. The methodof claim 12 wherein the stored information comprises propertyidentifiers indicative of conditions and the at least one processingdevice comprises an artificial intelligence module configured to comparethe signals indicative of the current condition to the stored propertyidentifiers and to determine the safety status of the person based onthe comparison.
 16. The method of claim 15 wherein the determining thesafety status comprises determining a position of a property identifieron a characteristic Bell curve based on the at least one signal.
 17. Themethod of claim 16 wherein the characteristic Bell curve is based onstored property identifiers indicative of the characteristic.
 18. Themethod of claim 16 wherein the characteristic Bell curve is related to:an object identifier; a location identifier; a position identifier; atime identifier; a sound identifier; a motion identifier; a physicalstatus identifier; an emotional identifier; or combinations thereof. 19.A non-transitory computer-readable medium whose contents configure asafety monitoring system to perform a method, the method comprising:receiving signals indicative of a current condition related to a safetystatus of a person; determining a safety status of the person based onthe signals indicative of the current condition and stored informationrelated to the safety status of the person; updating the storedinformation related to the safety status of the person based on thesignals indicative of the current condition; and when the determinedsafety status indicates the person may be in danger: initiating one ormore actions based on the determined safety status; monitoring responsesto the one or more initiated actions; and updating the storedinformation related to the safety status of the person based on themonitored responses.
 20. A system, comprising: means for generatingsignals indicative of a current condition related to a safety status ofa person; means for determining a safety status of the person based onthe signals indicative of the current condition and stored informationrelated to the safety status of the person; means for updating thestored information related to the safety status of the person based onthe signals indicative of the current condition; and means for, when thedetermined safety status indicates the person may be in danger,initiating one or more actions based on the determined safety status;monitoring responses to the one or more initiated actions; and updatingthe stored information related to the safety status of the person basedon the monitored responses.